Overcoming model bias for robust offline deep reinforcement learning

State-of-the-art reinforcement learning algorithms mostly rely on being allowed to directly interact with their environment collect millions of observations. This makes it hard transfer success industrial control problems, where simulations are often very costly or do not exist, and exploring in the real can potentially lead catastrophic events. Recently developed, model-free, offline RL algorithms, learn from a single dataset (containing limited exploration) by mitigating extrapolation error value functions. However, robustness training process is still comparatively low, problem known methods using To improve stability process, we use dynamics models assess policy performance instead functions, resulting MOOSE (MOdel-based Offline Search Ensembles), an algorithm which ensures low model bias keeping within support data. We compare state-of-the-art BRAC, BEAR BCQ Industrial Benchmark MuJoCo continuous tasks terms robust performance, find that outperforms its model-free counterparts almost all considered cases, even far.